Joint processing
Matthew Honnibal of SpaCy has commented that joint processing is the "motherhood and apple pie" of NLP. Indeed, everyone would agree that joint models should perform better than a cascade of independent models, not because they take more time and effort to build but because more information is available. Of course combining your models into one doesn't guarantee increased performance, as demonstrated in the CoNLL-2008 shared taskSurdeanu, M., & Johansson, R. (2008). The CoNLL-2008 shared task on joint parsing of syntactic and semantic dependencies. In Proceedings of the Twelfth Conference on Computational Natural Language Learning. Association for Computational Linguistics (pp. 159–177). and researchers have been exploring more and more methods to harness the power of joint processing. Tasks Multi-word expression recognition + parsing: Constant and Nivre (2016)Constant, M., & Nivre, J. (2016). A Transition-Based System for Joint Lexical and Syntactic Analysis. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 161–171). Association for Computational Linguistics. Retrieved from http://aclweb.org/anthology/P16-1016 POS tagging + parsing: Bohnet and Nivre (2012)Bohnet, Bernd, and Joakim Nivre. "A transition-based system for joint part-of-speech tagging and labeled non-projective dependency parsing."Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Association for Computational Linguistics, 2012., Zhenghua et al. (2014, for Chinese)Li, Zhenghua, et al. "Joint Optimization for Chinese POS Tagging and Dependency Parsing." IEEE/ACM Transactions on Audio, Speech, and Language Processing 22.1 (2014): 274-286., Nguyen et al. (2017)Nguyen, D. Q., Dras, M., & Johnson, M. (2017). A Novel Neural Network Model for Joint POS Tagging and Graph-based Dependency Parsing. arXiv Preprint arXiv:1705.05952. Word segmentation + POS tagging + parsing: Hatori et al. (2012)Hatori, Jun, et al. "Incremental joint approach to word segmentation, pos tagging, and dependency parsing in Chinese." Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1. Association for Computational Linguistics, 2012. Denis and Baldridge (2009)Denis, Pascal, and Jason Baldridge. "Global joint models for coreference resolution and named entity classification." Procesamiento del Lenguaje Natural 42, no. 1 (2009): 87-96. solve coreference resolution, anaphora resolution and named-entity classification in a joint global model. Constituent parsing + NER: * Finkel and Manning (2009)Finkel, J. R., & Manning, C. D. (2009). Joint parsing and named entity recognition. NAACL, 326–334. http://doi.org/10.3115/1620754.1620802: "Despite these earlier results, we found that combining parsing and named entity recognition modestly improved performance on both tasks. Our joint model produces an output which has consistent parse structure and named entity spans, and does a better job at both tasks than separate models with the same features." Entity Identification + Relation Extraction: Yu and Lam (2010)Yu, X., & Lam, W. (2010). Jointly Identifying Entities and Extracting Relations in Encyclopedia Text via a Graphical Model Approach. Proceedings of COLING ’10 Proceedings of the 23rd International Conference on Computational Linguistics (COLING 2010, (August), 1399–1407. Retrieved from http://dl.acm.org/citation.cfm?id=1944726, Yu et al. (2011)Yu, X., King, I., & Lyu, M. R. (2011). Towards a top-down and bottom-up bidirectional approach to joint information extraction. Proceedings of the 20th ACM International Conference on Information and Knowledge Management - CIKM ’11, (January), 847. http://doi.org/10.1145/2063576.2063699 CCG supertagging + Parsing: Auli and Lopez (2011)Auli, Michael, and Adam Lopez. "A comparison of loopy belief propagation and dual decomposition for integrated CCG supertagging and parsing."Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1. Association for Computational Linguistics, 2011. Entity coreference + typing + linking: Durrett and Klein (2014)Durrett, Greg, and Dan Klein. "A joint model for entity analysis: Coreference, typing, and linking." Transactions of the Association for Computational Linguistics 2 (2014): 477-490. Entity coreference + typing + relation: Singh et al. (2013)Singh, S., Riedel, S., Martin, B., Zheng, J., & McCallum, A. (2013). Joint Inference of Entities, Relations, and Coreference. Automated Knowledge Base Construction (AKBC CIKM 2013), 1–6. http://doi.org/10.1145/2509558.2509559 Entity coreference + linking: Hajishirzi et al. (2013)Hajishirzi, H., Zilles, L., Weld, D. S., & Zettlemoyer, L. (2013). Joint Coreference Resolution and Named-Entity Linking with Multi-pass Sieves. In EMNLP ’13 (pp. 289–299)., Dutta & Weikum (2015; cross-document)Dutta, Sourav, and Gerhard Weikum. "C3EL: A Joint Model for Cross-Document Co-Reference Resolution and Entity Linking." Conference on Empirical Methods in Natural Language Processing. ACL, 2015. Subtasks in Entity linking: mention detection + linking: From Ji et al. (2014)Ji, H., Nothman, J., & Hachey, B. (2014). Overview of TAC-KBP2014 Entity Discovery and Linking Tasks. TAC (Text Analysis Conference) 2014.: "Some recent work (Sil and Yates, 2013; Meij et al., 2012; Guo et al., 2013; Huang et al., 2014b) proved that mention extraction and mention linking can mutually enhance each other. Inspired by the these successes, many teams including IBM (Sil and Florian, 2014), MSIIPL THU (Zhao et al., 2014), SemLinker (Meurs et al., 2014), UBC (Barrena et al., 2014) and RPI (Hong et al., 2014) used the properties in external KBs such as DBPedia as feedback to refine the identification and classification of name mentions." Coreference resolution + mention head detection: Peng et al. (2015)Peng, H., Chang, K.-W., & Roth, D. (2015). A Joint Framework for Coreference Resolution and Mention Head Detection. CoNLL, 12–21. WSD + SRL: * Che and Liu (2010)Che, W., & Liu, T. (2010). Jointly Modeling WSD and SRL with Markov Logic. Proceedings of the 23rd International Conference on Computational Linguistics Coling 2010, (August), 161–169. Retrieved from http://www.aclweb.org/anthology/C10-1019: "We further propose a Markov logic model that jointly labels semantic roles and disambiguates all word senses. By evaluating our model on the OntoNotes 3.0 data, we show that this joint approach leads to a higher performance for word sense disambiguation and semantic role labeling than those pipeline approaches." * "The above idea, that the predicate senses and the semantic role labeling can help each other, may be inspired by Hajic et al. (2009), Surdeanu et al. (2008), and Dang and Palmer (2005). They have shown that semantic role features are helpful to disambiguate verb senses and vice versa." Named-entity recognition and linking: Luo et al. (2014)Luo, G., Huang, X., Lin, C., & Nie, Z. (2015). Joint Named Entity Recognition and Disambiguation. Emnlp, (September), 879–888. Event coref. + Temporal relation identification: Teng et al. (2016)Teng, Jiayue, Peifeng Li, Qiaoming Zhu, and Weiyi Ge. "Joint Event Co-reference Resolution and Temporal Relation Identification." In Chinese Lexical Semantics: 17th Workshop, CLSW 2016, Singapore, Singapore, May 20–22, 2016, Revised Selected Papers, vol. 10085, p. 426. Springer, 2016. Parsing+NMT: Eriguchi et al. (2017), Wu et al. (2017) See also * Category tree: Joint processing References *